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Abstract 


Different  theoretical  models  have  tried  to  investigate  the  feasibility  of  recurrent  neural  mechanisms 
for  achieving  direction  selectivity  in  the  visual  cortex.  The  mathematical  analysis  of  such  models  has 
been  restricted  so  far  to  the  case  of  purely  linear  networks.  We  present  an  exact  analytical  solution  of  the 
nonlinear  dynamics  of  a  class  of  direction  selective  recurrent  neural  models  with  threshold  nonlinearity. 
Our  mathematical  analysis  shows  that  such  networks  have  form-stable  stimulus-locked  traveling  pulse 
solutions  that  are  appropriate  for  modeling  the  responses  of  direction  selective  cortical  neurons.  Our 
analysis  shows  also  that  the  stability  of  such  solutions  can  break  down  giving  raise  to  a  different  class 
of  solutions  (’’lurching  activity  waves”)  that  are  characterized  by  a  specific  spatio-temporal  periodic¬ 
ity.  These  solutions  can  not  arise  in  models  for  direction  selectivity  with  purely  linear  spatio-temporal 
filtering. 
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1  Introduction 


Direction  selectivity  in  the  primary  visual  cortex  has  been  accounted  for  by  feed-forward  (e.g.  [10,  5,  11,  1]) 
as  well  as  by  recurrent  neural  mechanisms  [6,  7,  8].  The  mathematical  analysis  of  recurrent  mechanisms  for 
direction  selectivity  has  so  far  been  based  on  methods  from  linear  systems  theory  by  neglecting  the  nonlinear 
properties  of  the  neurons.  The  nonlinear  dynamic  phenomena  resulting  from  the  interplay  between  the 
recurrent  connectivity  and  the  nonlinear  threshold  characteristics  of  the  neurons  have  not  been  tractable  in 
this  theoretical  framework. 

In  this  paper  we  present  a  mathematical  analysis  that  takes  the  nonlinear  behavior  of  the  individual 
neurons  into  account.  We  present  the  result  of  an  analysis  of  networks  with  two  different  types  of  threshold 
nonlinearities  for  which  closed-form  analytical  solutions  of  the  network  dynamics  can  be  derived.  Our 
result  extends  previous  mathematical  work  on  the  the  dynamics  of  nonlinear  cortical  networks  with  strong 
recurrent  feedback  [2,  14,  4]  to  the  case  of  stimulus-driven  moving  solutions  in  networks  with  asymmetric 
lateral  connections. 

We  show  that  nonlinear  recurrent  networks  with  asymmetric  lateral  connections  have  a  class  of  form- 
stable  solutions,  in  the  following  signified  as  stimulus-locked  traveling  pulses,  that  are  suitable  for  modeling  the 
activity  of  direction  selective  neurons.  The  stability  of  the  traveling  pulse  solutions  of  the  nonlinear  network 
can  break  down  giving  raise  to  another  class  of  solutions  (lurching  activity  waves)  that  is  characterized 
by  spatio-temporal  periodicity.  Our  analysis  showed  that  networks  with  a  biologically  realistic  degrees  of 
direction  selectivity  typically  also  show  transitions  between  traveling  pulse  and  lurching  solutions. 

2  Basic  model 

Ensembles  of  direction  selective  neurons  with  the  same  receptive  field  center  and  preferred  speed  are  modeled 
as  points  of  a  continuous  neural  media,  or  neural  field.  Spatially  continuous  models  have  been  proposed  before 
to  analyze  the  dynamic  behavior  of  a  large  ensembles  of  neurons  in  the  visual  cortex  [12,  2,  9,  14].  The  scalar 
quantity  u{x,t )  represents  the  activity  of  neurons  with  receptive  field  center  x  at  time  t.  Its  dynamics  is 
given  by  the  integro-differential  equation: 

T  0  +  ”tM)  =  Iw(x  -  x'Wuix' ■  *»  + «*• 0  (1) 

The  function  /  is  nonlinear  and  characterizes  the  relationship  between  the  input  current  and  the  firing 
rate  of  the  individual  neuron  ensembles.  Two  different  forms  of  this  nonlinearity  are  discussed  below,  r 
is  a  time  constant  of  the  dynamics.  The  function  w(x)  characterizes  the  strength  of  the  excitatory  and 
inhibitory  lateral  connections,  dependent  on  the  positions  of  the  neurons  in  the  neural  field.  Direction 
selectivity  arises  when  this  function  is  asymmetric.  In  the  following  we  always  assume  that  the  stimulus 
has  a  constant  shape  that  is  translating  with  the  constant  stimulus  velocity  v,  leading  to  an  input  signal 
distribution  b(x,t)  =  B(x  —  vt )  in  the  neural  dynamics. 

To  analyze  the  dynamics  Eq.  (1)  we  introduce  a  new  coordinate  system  that  moves  together  with  the 
stimulus  by  defining  the  new  spatial  coordinate  y  =  x  —  vt  and  the  activity  distribution  U(y,t )  =  u(x,t)  (Cf. 
[14]).  This  results  in  the  dynamics: 

TWtu<yy'^  ~ TV '  dLdy^  +  U^y’^  =  j  w(y  ~ V  +  B(y)  (2) 

A  stationary  solution  in  the  moving  frame  corresponds  to  a  traveling  pulse  solution  with  velocity  v  in  the 
original  stationary  coordinate  system.  We  will  analyze  below  the  existence  and  stability  of  form-stable 
solutions  of  the  nonlinear  integro-differential  equation  Eq.  (2)  for  two  cases,  in  which  closed-form  solutions 
can  be  derived. 
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3  Analysis  for  step  function  threshold 


The  nonlinear  activation  function  is  taken  to  be  the  Heaviside  step  function  f(z)  =  Q(z),  where  Q(z)  =  1  for 
z  >  0,  and  zero  otherwise.  The  dynamics  with  a  constant  input  was  analyzed  by  Amari  [2],  whose  approach 
does  not  apply  directly  to  Eq.  (1)  because  of  the  presence  of  the  spatial  gradient.  We  assume  in  the  following 
the  existence  of  a  single  activation  peak,  and  we  signify  by  the  interval  I  =  [yi{t),y2{tj]  the  set  of  neurons 
with  non-negative  activation  ( U(y,t )  >  0).  Using  the  fact  that,  by  continuity,  U(yi,t)  =  U(y2,t )  =  0  we  can 
derive  the  following  self-consistent  equations  for  the  boundaries  of  the  interval  I  in  the  stationary  state 

W(yl-y*2)-W(  0)  =  B(y*)  (3) 

W(0)-W(yZ-yi)  =  B(y*2 )  (4) 

with  W(z)  =  w(y)dy  and  the  functions  W(y)  =  0\W{y)\rv]  and  B(y)  =  0[B(y)\rv],  where  O  is  the 
integral  operator  defined  by 


Uf™g(z')ex p(*=f)dz>  for  a  >  0 

0[g(z);a\  =  <  g(z }  for  a  =  0  (5) 

[  ir  S- oo  (Az')  exp(^)  d z'  for  a  <  0. 


The  stationary  solution  is  completely  defined  by  the  two  boundary  points  y{  and  y2  through  U*(y)  = 
W(y  —  y j)  —  W(y  —  y2)  +  B(y).  The  stability  of  this  solution  is  analyzed  by  perturbing  U*(y).  The  dynamics 
of  the  perturbation  SU(y,t)  is  given  by  the  linearized  dynamics 


M  _  TlJ^i  +  =  -fiLz3SWw,„  -  EkzpHlsuiyhl) 


dt 


dy 


(6) 


where  c*  =  dU*(yi)/dy  for  i  =  1,2  are  gradients  of  U*(y)  at  boundaries. 

By  solving  the  perturbation  dynamics  using  Laplace  transformation  it  can  be  shown  that  the  stationary 
solution  is  asymptotically  stable  if  the  following  equation 


(G{ 0, S)  -  4)(G( 0,  .s)  +  <■;)  =  G{y\  -  y*2,s)G(y*2  -  y^s)  (7) 

with  G{z,s)  =  0[w(z);tv/(  1  —  ts)]  has  no  solutions  in  the  open  right  half  of  the  complex  plane  (i.e.  for 
Re{s}  >  0). 


4  Analysis  for  half-wave  rectifying  threshold 

The  general  proceeding  used  in  section  3  can  in  principle  be  extended  for  networks  with  half-wave  rectifying 
threshold.  In  the  general  case,  however,  no  closed  form  solutions  can  be  derived  (See  [13]).  A  closed  form 
solution  can  be  obtained  by  extending  methods  from  [4].  Assume  that  the  network  is  defined  over  the 
periodic  spatial  domain  [— 7t/2,  7t/2]  .  When  the  stimulus  distribution  and  the  interaction  kernel  have  only 
a  small  number  of  Fourier  components,  a  closed  form  solution  can  be  derived.  We  analyzed  the  case  for 
B(x)  =  bo +  b2  cos(2(;r  —  xt,))  and  w(x)  =  wo  +w2  cos(2(;r  —  xw))  with  6,,  Wj,  and  xj,  and  xw  being  constants. 
In  this  case  the  dynamics  is  fully  determined  by  the  behavior  of  the  order  parameters 

W2  uy 

ro(t)  =  /  —u(x',t)  (8) 

/7r/2  r\r' 

—u(x',t)e2i{x-'s‘^n  (9) 

-7t/2 
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where  ’F(t)  is  chosen  as  to  keep  ro  real.  ro(t)  measures  the  mean  neural  activities  across  the  network  and  ro(t) 
signifies  the  spatial  modulation  in  the  activity  profile  of  u{x,t).  Different  from  Hansel’s  and  Sompolinsky’s 
model  [4],  the  network  we  analyzed  is  stimulus  driven  and  uses  an  asymmetric  interaction  kernel.  The  details 
of  the  mathematical  analysis  are  given  in  [13]. 

Figure  1  shows  the  comparison  between  the  results  from  the  mathematical  analysis  and  simulations. 
(The  network  with  step  threshold  leads  to  very  similar  simulation  results).  Panel  A  shows  the  speed  tuning 
curve  plotted  as  values  of  the  order  parameters  ro  and  ro  with  respect  to  different  stimulus  velocities  v. 
The  solid  lines  indicate  the  numerical  simulation  results,  and  the  dotted  lines  the  results  from  the  analytical 
solution.  For  both,  the  analytical  and  the  simulated  solution  the  velocity  tuning  curve  depends  critically 
on  the  asymmetrical  part  of  the  interaction  kernel.  The  asymmetry  determines  in  particular  the  optimal 
velocity  of  the  direction  selective  neurons  [13].  Panel  B  shows  the  largest  real  part  of  the  eigenvalues  of  a 
stability  matrix  that  can  be  obtained  by  linearizing  the  order  parameter  dynamics  around  the  stationary 
solution.  For  small  and  very  large  stimulus  velocities  the  largest  real  parts  of  the  eigenvalues  become  positive 
indicating  a  loss  of  stability  of  the  form-stable  solution.  To  verify  this  result  we  calculated  the  variances  of 
the  order  parameters  ro  and  ro  over  time  from  the  simulations.  Panel  C  shows  the  average  variations  as 
function  of  the  stimulus  velocity.  At  the  velocities  for  which  the  eigenvalues  indicate  a  loss  of  stability  the 
variability  of  the  amplitudes  suddenly  increases,  consistent  with  our  interpretation. 

An  interesting  observation  is  illustrated  in  panels  D  and  E  that  show  a  color-coded  plot  of  the  space-time 
evolution  of  the  activity.  Panel  E  shows  the  propagation  of  the  form-stable  peak  over  time.  Panel  D  shows 
the  solution  that  arises  when  stability  is  lost.  This  solution  is  characterized  by  a  spatio-temporal  periodicity 
that  is  defined  in  the  moving  coordinate  system  by  U (y ,t  +  nT0)  =  U(y,t ),  T0  being  a  constant  that  depends 
of  the  network  dynamics.  Solutions  of  this  type  have  been  described  before  in  spiking  thalamic  networks  [3]. 
We  found  that  this  solution  type  arises  very  robustly  for  both  types  of  threshold  functions  when  the  network 
achieved  substantial  direction  selective  behavior. 


5  Conclusion 

We  have  presented  different  methods  for  an  analysis  of  the  nonlinear  dynamics  of  simple  recurrent  neural 
models  for  the  direction  selectivity  of  cortical  neurons.  Compared  to  earlier  work,  we  have  taken  into  account 
the  essentially  nonlinear  effects  that  are  introduced  by  the  nonlinear  threshold  characteristics  of  the  cortical 
neurons.  The  key  result  of  our  work  is  that  such  networks  have  a  class  of  form-stable  traveling  pulse  solutions 
that  behave  similar  as  the  solutions  of  linear  spatio-temporal  filtering  models  within  a  certain  regime  of 
stimulus  speeds.  By  the  essential  nonlinearity  of  the  network,  however,  bifurcations  can  arise  for  which  the 
traveling  pulse  solutions  become  unstable.  We  observed  that  in  this  case  a  new  class  of  spatio-temporally 
periodic  solutions  (’’lurching  activity  waves”)  arises.  Since  we  found  this  solution  type  very  frequently  for 
networks  with  substantial  direction  selectivity  our  analysis  suggests  that  such  ’’lurching  behavior”  might  be 
observable  in  the  primary  visual  cortex,  providing  strong  evidence  for  the  hypothesis  that  direction  selectivity 
is  essentially  based  on  lateral  connectivity.  An  experimental  demonstration  of  this  effect  might,  however,  be 
complicated  by  the  mutual  inhibition  of  neuron  ensembles  that  are  selective  for  different  stimulus  speeds. 

Acknowledgements  We  thank  H.S.  Seung  and  T.  Poggio  for  very  helpful  comments. 
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Figure  1:  Traveling  pulse  solution  and  its  stability  for  the  half-wave  rectifying  threshold  model.  Panel  A 
shows  the  velocity  tuning  curves  of  the  order  parameters  ro  and  rn-  The  solid  lines  indicate  the  results 
from  the  numerical  simulation,  while  dotted  lines  represent  the  results  from  the  analytical  solution.  Panel  B 
shows  the  maximum  real  part  of  the  eigenvalues  of  a  stability  matrix  that  can  be  obtained  from  perturbed 
linear  dynamics  around  the  stationary  solution.  For  small  and  very  large  stimulus  velocities  the  largest  real 
part  of  the  eigenvalues  becomes  positive  indicating  a  loss  of  stability  of  the  form-stable  solution.  Panel  C 
shows  the  average  variances  over  time  of  the  order  parameters  ro  (blue  curve)  and  r-2  (green  curve)  obtained 
from  the  simulation.  A  nonzero  variance  signifies  a  loss  of  stability  of  the  traveling  pulse  solution.  This 
result  is  consistent  with  eigenvalue  analysis  in  Panel  B.  A  color  coded  plot  of  spatial-temporal  evolution  of 
the  activity  u(x,t)  is  shown  in  panels  D  and  E.  Panel  E  shows  the  propagation  of  the  form-stable  peak  over 
time.  Panel  D  shows  the  lurching  activity  wave  that  arises  when  stability  is  lost. 
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